For flame recognition problem, the traditional recognition methods based on physical signal are easily affected by the external environment. Meanwhile, most of the current methods based on feature extraction of flame image are less discriminative to different scene and flame type, and then have lower recognition precision if the flame scene and type change. To overcome this drawback, a new fast recognition method for flame image was proposed by introducing colorspace information into Scale Invariant Feature Transform (SIFT) algorithm. Firstly, the feature descriptors of flame were extracted by SIFT algorithm from the frame images which were obtained from flame video. Secondly, the local noisy feature points were filtered by introducing the feature information of flame colorspace, and the feature descriptors were transformed into feature vectors by means of Bag Of Keypoints (BOK). Finally, Extreme Learning Machine (ELM) was utilized to establish a fast flame recognition model. Experiments were conducted on open flame datasets and real-life flame images. The results show that for different flame scenes and types the accuracy of the proposed method is more than 97%, and the recognition time is just 2.19 s for test set which contains 4301 images. In addition, comparing with the other three methods such as support vector machine based on entropy, texture and flame spread rate, support vector machine based on SIFT and fire specialty in color space, ELM based on SIFT and fire specialty in color space, the proposed method outperforms in terms of recognition accuracy and speed.